A parallel workload has extreme variability in a production environment

نویسندگان

  • R. Henwood
  • N. W. Watkins
  • S. C. Chapman
  • R. McLay
چکیده

Writing data in parallel is a common operation in some computing environments and a good proxy for a number of other parallel processing patterns. The duration of time taken to write data in large-scale compute environments can vary considerably. This variation comes from a number of sources, both systematic and transient. The result is a highly complex behavior that is difficult to characterize. This paper further develops the model for parallel task variability proposed in the paper “A parallel workload has extreme variability” (Henwood et. al 2016). This model is the Generalized Extreme Value (GEV) distribution. This paper further develops the systematic analysis that leads to the GEV model with the addition of a traffic congestion term. Observations of a parallel workload are presented from a High Performance Computing environment under typical production conditions, which include traffic congestion. An analysis of the workload is performed and shows the variability tends towards GEV as the order of parallelism is increased. The results are presented in the context of Amdahl’s law and the predictive properties of a GEV models are discussed. A optimization for certain machine designs is also suggested.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Understanding the Causes of Performance Variability in HPC Applications

While most workload characterization focuses on application and architecture performance, the variability in performance also has wide ranging impacts on the users and managers of large scale computing resources. Performance variability, while secondary to absolute or optimal performance itself, can significantly detract from both the overall performance realized by parallel workloads and the s...

متن کامل

A parallel workload has extreme variability

In both high-performance computing (HPC) environments and the public cloud, the duration of time to retrieve or save your results is simultaneously unpredictable and important to your over all resource budget. It is generally accepted (“Google: Taming the Long Latency Tail When More Machines Equals Worse Results”, Todd Hoff, highscalability.com 2012) , but without a robust explanation, that ide...

متن کامل

Dynamic File-access Characteristics of a Production Parallel Scientiic Workload

Multiprocessors have permitted astounding increases in computational performance, but many cannot meet the intense I/O requirements of some scientiic applications. An important component of any solution to this I/O bottleneck is a parallel le system that can provide high-bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors. Most successful systems ar...

متن کامل

Characteristics of a Production Parallel Scienti c Workload

Multiprocessors have permitted astounding increases in computational performance but many cannot meet the intense I O requirements of some scienti c applications An important component of any solution to this I O bottleneck is a parallel le system that can provide high bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors Most successful systems are b...

متن کامل

Creating Full Envelopment in Data Envelopment Analysis with Variable Returns to Scale Technology

In this paper, weak defining hyperplanes and the anchor points in DEA, as an important subset of the set of extreme efficient points of the Production Possibility Set (PPS), are used to construct unobserved DMUs and in the long run to improve the envelopment of all observed DMUs. There has been a surge of articles on improving envelopment in recent years. What has been done first is in Constant...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1801.03898  شماره 

صفحات  -

تاریخ انتشار 2018